Conversation
ea4740c to
4554267
Compare
Clients that expose a model picker can inject thinking defaults that conflict with the mode a user selected. --reasoning/--thinking set the server-side default used when a request omits one, --reasoning-effort picks the default effort, and --alias advertises an extra model id in the model catalogue so a picker can target this instance by name. Alias suffixes select the default thinking mode: -chat/-nothink disable it and -reasoner enables it. Match on the suffix rather than the whole string, so a local quant name such as local-glm-5.3-flash-q2 stays unaffected while a ...-chat alias still opts out. Turning the default off only flips the boolean. Setting the effort to NONE as well would pin think_mode_from_enabled() to NONE, so a client that opts back in through a -reasoner alias or an explicit thinking:true was silently ignored. Keep the configured effort so opting in still yields a usable mode; an explicit --reasoning-effort none remains a deliberate opt-out. While touching the catalogue, fix the ids it advertises. GLM-5.3 and GLM-5.2 share DS4_MODEL_FAMILY_GLM_DSA, so a family test made a 5.3 engine list glm-5.2 and misreport the loaded model to a picker. Select the ids by variant, and drive both the catalogue and the alias-deduplication check from that one list so a built-in alias is never advertised twice. Rebased onto 8db1d1d. Two conflicts came from changes that landed in the same places since this was written, and neither side was dropped: * Both sides extended model_alias_{disables,enables}_thinking() -- upstream by listing the qwen3.8 aliases, this branch by matching suffixes. The suffix rules and the full exact lists are both present, so the aliases upstream added resolve exactly as they did before. * send_models() was rewritten on both sides. This branch's shape wins, since driving one list is the point, but its table only knew glm53/glm52/deepseek: a Qwen3.8 or V4.1 engine would have been advertised as deepseek-v4-flash. The table now covers both, in the order the engine-specific branches used, and test_model_catalog_matches_loaded_variant pins all five families.
4554267 to
239b8a3
Compare
|
Rebased onto Conflict 1 — Conflict 2 —
|
Motivation
Clients that expose a model picker can inject thinking defaults that conflict
with the mode a user selected. This adds server-side defaults plus a way to
advertise an instance under a custom model id.
Changes
--reasoning on|off/--thinking on|off— default thinking state when arequest omits one. The two are synonyms; the error message reports the flag
the user actually typed.
--no-thinking— shortcut for--reasoning off.--reasoning-effort none|minimal|low|medium|high|xhigh|max— default effort.--alias NAME— advertise an extra model id in the catalogue so a picker cantarget this instance by name. The id is also served by
GET /v1/models/NAME.Alias suffixes select the default thinking mode:
-chat/-nothinkdisable it,-reasonerenables it. The match is on the suffix rather than a substring, so alocal quant name such as
local-glm-5.3-flash-q2stays unaffected while a...-chatalias still opts out.Turning the default off must not disable thinking permanently
think_mode_from_enabled()returnsNONEwhenever the effort isNONE, so anearlier revision that set
default_think_mode = NONEon--reasoning offpinned every request to non-thinking. A client that opted back in through a
-reasoneralias or an explicitthinking: truewas silently ignored.Off now flips only the boolean and keeps the configured effort:
Catalogue bug fix
send_models()picked the advertised ids with a family test, but GLM-5.3 andGLM-5.2 share
DS4_MODEL_FAMILY_GLM_DSA. A GLM-5.3 engine therefore advertisedglm-5.2,glm-5.2-chatandglm-5.2-reasoner, misreporting the loaded modelto a picker. The ids are now selected by variant, and a single list drives both
the catalogue and the alias-deduplication check.
Testing
Machine: Apple M4, macOS 26.4.1, 16 GB. Backend: Metal (default build).
--servercovers the request parsing, chat rendering and thinking-control pathsthis change touches. It needs no model file. The model-backed suites
(
--logprob-vectors,--long-context, ...) were not run because this machinehas no GGUF checked out.
New tests:
test_server_reasoning_options— flag parsing, the off/on effort combinationsabove, plus negative alias-suffix cases (
my-chat-model,local-chat-assistantmust not disable thinking).test_model_catalog_matches_loaded_variant— the three variant catalogues,and every advertised id is accepted by
server_model_alias_known().test_send_models_json_shape— drives the real/v1/modelsbody over asocketpair and checks the JSON shape, plus that a built-in alias is not
advertised twice.
No inference-backend code is touched, so no speed regression is expected.